From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue

Emotion recognition in conversations (ERC) is a crucial task for building human-like conversational agents. While substantial efforts have been devoted to ERC for chit-chat dialogues, the task-oriented counterpart is largely left unattended. Directly applying chit-chat ERC models to task-oriented dialogues (ToDs) results in suboptimal performance as these models overlook key features such as the correlation between emotions and task completion in ToDs. In this paper, we propose a framework that turns a chit-chat ERC model into a task-oriented one, addressing three critical aspects: data, features and objective. First, we devise two ways of augmenting rare emotions to improve ERC performance. Second, we use dialogue states as auxiliary features to incorporate key information from the goal of the user. Lastly, we leverage a multi-aspect emotion definition in ToDs to devise a multi-task learning objective and a novel emotion-distance weighted loss function. Our framework yields significant improvements for a range of chit-chat ERC models on EmoWOZ, a large-scale dataset for user emotion in ToDs. We further investigate the generalisability of the best resulting model to predict user satisfaction in different ToD datasets. A comparison with supervised baselines shows a strong zero-shot capability, highlighting the potential usage of our framework in wider scenarios.Comment: Accepted by SIGDIAL 202

EmoUS: Simulating User Emotions in Task-Oriented Dialogues

Existing user simulators (USs) for task-oriented dialogue systems only model user behaviour on semantic and natural language levels without considering the user persona and emotions. Optimising dialogue systems with generic user policies, which cannot model diverse user behaviour driven by different emotional states, may result in a high drop-off rate when deployed in the real world. Thus, we present EmoUS, a user simulator that learns to simulate user emotions alongside user behaviour. EmoUS generates user emotions, semantic actions, and natural language responses based on the user goal, the dialogue history, and the user persona. By analysing what kind of system behaviour elicits what kind of user emotions, we show that EmoUS can be used as a probe to evaluate a variety of dialogue systems and in particular their effect on the user's emotional state. Developing such methods is important in the age of large language model chat-bots and rising ethical concerns.Comment: accepted by SIGIR202

CAMELL: Confidence-based Acquisition Model for Efficient Self-supervised Active Learning with Label Validation

Supervised neural approaches are hindered by their dependence on large, meticulously annotated datasets, a requirement that is particularly cumbersome for sequential tasks. The quality of annotations tends to deteriorate with the transition from expert-based to crowd-sourced labelling. To address these challenges, we present \textbf{CAMELL} (Confidence-based Acquisition Model for Efficient self-supervised active Learning with Label validation), a pool-based active learning framework tailored for sequential multi-output problems. CAMELL possesses three core features: (1) it requires expert annotators to label only a fraction of a chosen sequence, (2) it facilitates self-supervision for the remainder of the sequence, and (3) it employs a label validation mechanism to prevent erroneous labels from contaminating the dataset and harming model performance. We evaluate CAMELL on sequential tasks, with a special emphasis on dialogue belief tracking, a task plagued by the constraints of limited and noisy datasets. Our experiments demonstrate that CAMELL outperforms the baselines in terms of efficiency. Furthermore, the data corrections suggested by our method contribute to an overall improvement in the quality of the resulting datasets

ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?

Recent research on dialogue state tracking (DST) focuses on methods that allow few- and zero-shot transfer to new domains or schemas. However, performance gains heavily depend on aggressive data augmentation and fine-tuning of ever larger language model based architectures. In contrast, general purpose language models, trained on large amounts of diverse data, hold the promise of solving any kind of task without task-specific training. We present preliminary experimental results on the ChatGPT research preview, showing that ChatGPT achieves state-of-the-art performance in zero-shot DST. Despite our findings, we argue that properties inherent to general purpose models limit their ability to replace specialized systems. We further theorize that the in-context learning capabilities of such models will likely become powerful tools to support the development of dedicated and dynamic dialogue state trackers.Comment: 13 pages, 3 figures, accepted at ACL 202

Results of the COVID-19 mental health international for the general population (COMET-G) study.

INTRODUCTION: There are few published empirical data on the effects of COVID-19 on mental health, and until now, there is no large international study. MATERIAL AND METHODS: During the COVID-19 pandemic, an online questionnaire gathered data from 55,589 participants from 40 countries (64.85% females aged 35.80 ± 13.61; 34.05% males aged 34.90±13.29 and 1.10% other aged 31.64±13.15). Distress and probable depression were identified with the use of a previously developed cut-off and algorithm respectively. STATISTICAL ANALYSIS: Descriptive statistics were calculated. Chi-square tests, multiple forward stepwise linear regression analyses and Factorial Analysis of Variance (ANOVA) tested relations among variables. RESULTS: Probable depression was detected in 17.80% and distress in 16.71%. A significant percentage reported a deterioration in mental state, family dynamics and everyday lifestyle. Persons with a history of mental disorders had higher rates of current depression (31.82% vs. 13.07%). At least half of participants were accepting (at least to a moderate degree) a non-bizarre conspiracy. The highest Relative Risk (RR) to develop depression was associated with history of Bipolar disorder and self-harm/attempts (RR = 5.88). Suicidality was not increased in persons without a history of any mental disorder. Based on these results a model was developed. CONCLUSIONS: The final model revealed multiple vulnerabilities and an interplay leading from simple anxiety to probable depression and suicidality through distress. This could be of practical utility since many of these factors are modifiable. Future research and interventions should specifically focus on them

Results of the COVID-19 mental health international for the health professionals (COMET-HP) study: depression, suicidal tendencies and conspiracism.

INTRODUCTION: The current study aimed to investigate the rates of anxiety, clinical depression, and suicidality and their changes in health professionals during the COVID-19 outbreak. MATERIALS AND METHODS: The data came from the larger COMET-G study. The study sample includes 12,792 health professionals from 40 countries (62.40% women aged 39.76 ± 11.70; 36.81% men aged 35.91 ± 11.00 and 0.78% non-binary gender aged 35.15 ± 13.03). Distress and clinical depression were identified with the use of a previously developed cut-off and algorithm, respectively. STATISTICAL ANALYSIS: Descriptive statistics were calculated. Chi-square tests, multiple forward stepwise linear regression analyses, and Factorial Analysis of Variance (ANOVA) tested relations among variables. RESULTS: Clinical depression was detected in 13.16% with male doctors and 'non-binary genders' having the lowest rates (7.89 and 5.88% respectively) and 'non-binary gender' nurses and administrative staff had the highest (37.50%); distress was present in 15.19%. A significant percentage reported a deterioration in mental state, family dynamics, and everyday lifestyle. Persons with a history of mental disorders had higher rates of current depression (24.64% vs. 9.62%; p < 0.0001). Suicidal tendencies were at least doubled in terms of RASS scores. Approximately one-third of participants were accepting (at least to a moderate degree) a non-bizarre conspiracy. The highest Relative Risk (RR) to develop clinical depression was associated with a history of Bipolar disorder (RR = 4.23). CONCLUSIONS: The current study reported findings in health care professionals similar in magnitude and quality to those reported earlier in the general population although rates of clinical depression, suicidal tendencies, and adherence to conspiracy theories were much lower. However, the general model of factors interplay seems to be the same and this could be of practical utility since many of these factors are modifiable

The effect of different degrees of lockdown and self-identified gender on anxiety, depression and suicidality during the COVID-19 pandemic: Data from the international COMET-G study.

INTRODUCTION: During the COVID-19 pandemic various degrees of lockdown were applied by countries around the world. It is considered that such measures have an adverse effect on mental health but the relationship of measure intensity with the mental health effect has not been thoroughly studied. Here we report data from the larger COMET-G study pertaining to this question. MATERIAL AND METHODS: During the COVID-19 pandemic, data were gathered with an online questionnaire from 55,589 participants from 40 countries (64.85% females aged 35.80 ± 13.61; 34.05% males aged 34.90±13.29 and 1.10% other aged 31.64±13.15). Anxiety was measured with the STAI, depression with the CES-D and suicidality with the RASS. Distress and probable depression were identified with the use of a previously developed cut-off and algorithm respectively. STATISTICAL ANALYSIS: It included the calculation of Relative Risk (RR), Factorial ANOVA and Multiple backwards stepwise linear regression analysis RESULTS: Approximately two-thirds were currently living under significant restrictions due to lockdown. For both males and females the risk to develop clinical depression correlated significantly with each and every level of increasing lockdown degree (RR 1.72 and 1.90 respectively). The combined lockdown and psychiatric history increased RR to 6.88 The overall relationship of lockdown with severity of depression, though significant was small. CONCLUSIONS: The current study is the first which reports an almost linear relationship between lockdown degree and effect in mental health. Our findings, support previous suggestions concerning the need for a proactive targeted intervention to protect mental health more specifically in vulnerable groups

Results of the COVID-19 mental health international for the health professionals (COMET-HP) study : depression, suicidal tendencies and conspiracism

Publisher Copyright: © 2023, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany.Introduction: The current study aimed to investigate the rates of anxiety, clinical depression, and suicidality and their changes in health professionals during the COVID-19 outbreak. Materials and methods: The data came from the larger COMET-G study. The study sample includes 12,792 health professionals from 40 countries (62.40% women aged 39.76 ± 11.70; 36.81% men aged 35.91 ± 11.00 and 0.78% non-binary gender aged 35.15 ± 13.03). Distress and clinical depression were identified with the use of a previously developed cut-off and algorithm, respectively. Statistical analysis: Descriptive statistics were calculated. Chi-square tests, multiple forward stepwise linear regression analyses, and Factorial Analysis of Variance (ANOVA) tested relations among variables. Results: Clinical depression was detected in 13.16% with male doctors and ‘non-binary genders’ having the lowest rates (7.89 and 5.88% respectively) and ‘non-binary gender’ nurses and administrative staff had the highest (37.50%); distress was present in 15.19%. A significant percentage reported a deterioration in mental state, family dynamics, and everyday lifestyle. Persons with a history of mental disorders had higher rates of current depression (24.64% vs. 9.62%; p < 0.0001). Suicidal tendencies were at least doubled in terms of RASS scores. Approximately one-third of participants were accepting (at least to a moderate degree) a non-bizarre conspiracy. The highest Relative Risk (RR) to develop clinical depression was associated with a history of Bipolar disorder (RR = 4.23). Conclusions: The current study reported findings in health care professionals similar in magnitude and quality to those reported earlier in the general population although rates of clinical depression, suicidal tendencies, and adherence to conspiracy theories were much lower. However, the general model of factors interplay seems to be the same and this could be of practical utility since many of these factors are modifiable.Peer reviewe